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Accountability  *  Integrity  *  Reliability 


United  States  General  Accounting  Office 
Washington,  DC  20548 


August  31,  2001 

The  Honorable  Ernest  F.  Hollings 
Chairman,  Committee  on  Commerce, 

Science,  and  Transportation 
United  States  Senate 

The  Honorable  John  D.  Rockefeller  IV 
Chairman,  Subcommittee  on  Aviation 
Committee  on  Commerce,  Science, 
and  Transportation 
United  States  Senate 

To  help  meet  the  growing  demand  for  air  travel,  the  Federal  Aviation 
Administration  (FAA),  in  collaboration  with  the  aviation  community,  is 
implementing  a  new  approach  for  air  traffic  management  known  as  free 
flight.  Under  this  approach,  FAA  is  moving  gradually  from  its  present  use 
of  highly  structured  rules  and  procedures  for  air  traffic  operations  to  a 
more  flexible  approach,  which  increases  collaboration  between  FAA  and 
the  aviation  community.  By  using  a  set  of  new  automated  technologies 
(tools)  and  procedures,  free  flight  is  intended  to  increase  the  capacity  and 
efficiency  of  our  nation’s  airspace  system  while  helping  to  minimize 
delays.  Two  of  these  tools,  the  Traffic  Management  Advisor  and  the 
passive  Final  Approach  Spacing  Tool,  provide  controllers  with  a  more 
efficient  and  effective  means  to  increase  the  capacity  of  our  nation’s 
airspace  system  by  better  scheduling,  sequencing,  spacing,  and  assigning 
aircraft  to  runways.  These  two  tools  are  expected  to  allow  more  aircraft  to 
land  during  peak  periods  of  traffic,  thus  increasing  capacity  and 
minimizing  delays.  Another  tool,  the  User  Request  Evaluation  Tool,  allows 
controllers  to  make  more  efficient  use  of  the  existing  airspace  by  allowing 
aircraft  to  fly  optimal  or  more  direct  routes,  thus  helping  to  reduce  delays 
at  major  airports.1  Collectively,  these  tools  are  also  designed  to  achieve  the 
above  benefits  without  negatively  affecting  safety. 


^ur  review  focused  on  these  three  tools  because  they  account  for  80  percent  of  the  budget 
for  phases  1  and  2.  We  did  not  review  two  other  automated  tools — the  Surface  Movement 
Advisor  and  the  Collaborative  Decision  Making — because  generally  they  had  been 
implemented  at  all  phase  1  locations  before  we  started  this  review,  and  FAA  plans  to 
include  additional  capabilities  in  phase  2. 
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In  1998,  we  issued  a  report  that  identified  the  challenges  facing  FAA  in  its 
efforts  to  implement  free  flight  in  a  cost-effective  manner.2  Since  that 
report,  FAA  has  moved  aggressively  with  its  phased  approach  to  deploying 
free  flight  tools  to  gain  operational  experience  before  making  a  total 
commitment  to  the  program.  During  the  first  phase,  FAA  has  been 
deploying  the  new  tools  to  selected  locations  generally  on  or  ahead  of 
schedule  and  has  been  measuring  their  benefits.  Under  the  schedule 
developed  with  the  aviation  community,  FAA  plans  to  complete  this  first 
phase  by  the  end  of  calendar  year  2002,  at  a  cost  of  about  $630  million.  In 
March  2002,  the  agency  plans  to  make  an  investment  decision — estimated 
at  $717  million— on  whether  to  move  to  the  next  phase,3  which  would 
result  in  a  long-term  funding  commitment  from  FAA  as  part  of  its  plan  to 
meet  the  nation’s  air  transportation  needs.4 

Because  of  the  importance  of  the  free  flight  program  to  the  future 
operation  of  our  nation’s  airspace  system  and  FAA’s  upcoming  investment 
decision,  the  Chairmen  of  the  Committee  on  Commerce,  Science,  and 
Transportation  and  the  Subcommittee  on  Aviation  asked  us  to  review  the 
program’s  status  to  help  them  determine  whether  FAA  will  be  in  a  position 
by  March  2002  to  make  a  decision  to  move  to  the  next  phase.  This  report 
discusses  (1)  the  technical  and  operational  issues  that  could  impair  the 
ability  of  the  free  flight  tools  to  achieve  their  full  potential  and  (2)  the 
extent  to  which  these  tools  will  increase  capacity  and  efficiency  while 
helping  to  minimize  delays  in  our  nation’s  airspace  system.  To  determine 
the  technical  and  operational  issues  associated  with  the  tools,  we  used 
recognized  best  practice  criteria  for  developing,  managing,  and  evaluating 
software  projects.  To  gain  further  insight  into  these  issues,  as  well  as  the 
tools’  benefits,  we  held  formal  discussion  group  meetings  with  controllers, 
maintenance  technicians,  and  supervisors  who  were  part  of  nationwide 
user  teams  for  one  of  the  tools  and  interviewed  individual  users  for  the 
other  two  tools.  In  addition,  we  interviewed  program  and  contractor 
officials  and  examined  documentation  on  the  goals  and  benefits  from 


2National  Airspace  System:  FAA  Has  Implemented  Some  Free  Flight  Initiatives ,  but 
Challenges  Remain  (GAO/RCED-98-246,  Sept.  28, 1998). 

3The  estimated  investment  of  $717  million  is  the  total  development  costs  for  fiscal  years 
2001  through  2010. 

4In  June  2001,  FAA  released  its  10-year  Opera  tional  Evolution  Plan ,  which  identifies  near- 
term  (2001),  mid-term  (2002  through  2004),  and  long-term  (2005  through  2010)  plans  for 
increasing  efficiency  and  capacity,  managing  delays,  and  maintaining  safety.  FAA 
anticipates  completing  phase  1  and  beginning  phase  2  of  its  free  flight  program  in  its  near- 
and  mid-term  plans. 
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these  tools.  Appendix  I  provides  additional  details  on  our  scope  and 
methodology. 


Results  in  Brief 


Collectively  the  free  flight  tools  have  demonstrated  some  benefits.  By 
March  2002,  FAA  should  have  sufficient  information  to  make  an 
investment  decision  about  deploying  the  Traffic  Management  Advisor  tool 
to  additional  sites  in  phase  2.  However,  the  agency’s  current  deployment 
schedule  might  not  allow  it  to  collect  sufficient  data  to  make  a  similar 
determination  about  the  further  deployment  of  the  User  Request 
Evaluation  Tool.  Because  of  the  risks  associated  with  the  passive  Final 
Approach  Spacing  Tool,  FAA  has  decided  not  to  deploy  this  tool  in  phase 
2.  FAA  is  addressing  technical  and  operational  issues  that  could  affect  the 
ability  of  these  tools  to  achieve  their  full  capabilities.  The  major  technical 
challenge  is  ensuring  that  the  User  Request  Evaluation  Tool  can  effectively 
work  with  other  key  air  traffic  control  systems.  While  FAA  has  been 
testing  this  tool  to  ensure  that  it  effectively  works  with  other  key  systems, 
this  testing  has  not  yet  been  completed.  Ensuring  that  URET  works  as 
intended  is  important  so  FAA  can  fully  realize  the  tool's  benefit  of 
increasing  controllers'  productivity.  We  are  recommending  that  FAA 
collect  and  analyze  data  to  better  ensure  that  the  technical  issue  with  this 
tool  is  resolved  before  moving  forward. 

As  for  operational  issues,  FAA’s  greatest  challenge  is  effectively  managing 
the  cultural  changes  in  controllers’  roles  and  responsibilities  that  using  the 
free  flight  tools  entails  and  communicating  the  capabilities  of  the  Traffic 
Management  Advisory  tool  to  controllers.  The  program's  heavy  reliance 
on  technology  will  require  them  to  shift  from  their  traditional  ways  of 
managing  air  traffic.  For  example,  most  controllers  rely  on  their 
professional  judgment  to  sequence  properly  separated  aircraft  that  are 
starting  to  descend  into  the  airspace  near  an  airport.  To  sequence  an 
aircraft’s  descent  with  the  Traffic  Management  Advisor  tool,  controllers 
will  rely  on  the  computer  to  assign  a  certain  time  for  that  aircraft  to  arrive 
at  a  predetermined  point  in  the  airspace  near  an  airport.  Training  in  free 
flight’s  new  tools  is  integral  to  air  traffic  controllers’  and  other  users’ 
fulfilling  their  new  roles  and  responsibilities  as  well  as  developing  their 
confidence  in  using  new  technologies.  Although  users  have  indicated  that 
they  lack  the  specialized  expertise  to  develop  training,  FAA  has  been 
relying  primarily  upon  them  to  develop  and  present  it.  We  are  making  a 
recommendation  for  FAA  to  improve  the  training  being  developed  and 
provided  so  users  can  become  proficient  with  these  new  tools.  As  for  the 
Traffic  Management  Advisor  tool,  users  expressed  uncertainties  regarding 
what  capabilities  the  tool  will  provide  because  FAA  has  not  communicated 


Page  3 


GAO-0 1-932  FAA’s  Free  Flight  Program 


this  information  to  them.  As  a  result,  it  is  difficult  for  users  to  know  what 
to  expect  and  for  FAA  to  evaluate  benefits. 

Preliminary  data  have  indicated  that  the  free  flight  tools  have  the  potential 
to  increase  both  capacity  and  efficiency.  However,  because  the  future 
demand  for  air  traffic  services  is  expected  to  outpace  the  increases 
expected  from  the  tools,  the  collective  length  of  the  delays  during  peak 
periods  will  continue  to  increase  but  not  to  the  extent  they  would  have 
without  them.  For  example,  FAA’s  analyses  indicate  that  the  collective 
length  of  delays  would  increase  by  about  one-half  hour  with  the  tools  as 
opposed  to  by  1  hour  without  the  tools.  In  May  2001,  FAA  established 
quantifiable  goals  for  increasing  system  capacity — ranging  from  3  to  5 
percent — for  two  of  the  three  free  flight  tools.  FAA’s  preliminary  data 
show  that  the  Traffic  Management  Advisor  and  the  passive  Final  Approach 
Spacing  tools  have  met  some  of  the  capacity  goals,  and  the  agency  expects 
data  from  additional  sites  to  produce  similar  results  so  that,  by  March 
2002,  it  will  have  sufficient  data  to  support  the  decision  to  move  forward. 
The  goal  for  the  third  tool — the  User  Request  Evaluation  Tool — is  to 
increase  system  efficiency  (by  using  direct  routings)  by  15  percent  within 
the  first  year  of  being  fully  implemented,  which  reduces  flight  time  and 
fuel  costs  for  the  airlines.  Although  FAA’s  data  demonstrate  that  this  tool 
has  met  this  objective  at  two  facilities  and  collectively  saved  the  airlines 
approximately  $1.5  million  per  month  in  operating  costs,  the  data  are 
based  on  the  prototype  system.  While  FAA  states  that  the  actual  system 
will  essentially  have  the  same  capabilities  as  its  prototype,  until  that 
system  is  deployed  and  its  benefits  measured  the  extent  of  these  benefits 
remain  uncertain.  Moreover,  the  benefits  that  FAA  has  documented  for 
using  this  tool  reflect  savings  for  a  segment  of  a  flight — when  an  airplane 
is  traveling  through  high-altitude  airspace — not  the  entire  flight  from 
departure  to  arrival.  Consequently,  these  savings  could  be  diminished  as 
an  airplane  flies  into  more  congested  airspace.  We  are  making 
recommendations  to  ensure  that  FAA’s  goals  for  these  tools  are  worth  the 
investment. 


Background 


Recent  advances  in  aircraft  technology,  including  advanced  collision 
avoidance  and  flight  management  systems,  and  new  automated  tools  for 
air  traffic  controllers  enable  a  shift  from  air  traffic  control  to  collaborative 
air  traffic  management.  Free  flight,  a  key  component  of  air  traffic 
management,  will  provide  pilots  with  more  flexibility,  under  certain 
conditions,  to  fly  more  direct  routes  from  city  to  city.  Currently,  pilots 
primarily  fly  fixed  routes — the  aerial  equivalent  of  the  interstate  highway 
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system — that  often  are  less  direct  because  pilots  are  dependent  on  ground- 
based  navigational  aids. 

Through  free  flight,  FAA  hopes  to  increase  the  capacity,  efficiency,  and 
safety  of  our  nation's  airspace  system  to  meet  the  growing  demand  for  air 
transportation  as  well  as  enhance  the  controllers’  productivity.  The 
aviation  industry,  especially  the  airlines,  is  seeking  to  shorten  flight  times 
and  reduce  fuel  consumption.  According  to  FAA’s  preliminary  estimates, 
the  benefits  to  the  flying  public  and  the  aviation  industry  could  reach  into 
the  billions  of  dollars  when  the  program  is  fully  operational. 

In  1998,  FAA  and  the  aviation  community  agreed  to  a  phased  approach  for 
implementing  the  free  flight  program,  established  a  schedule  for  phase  1, 
and  created  a  special  program  office  to  manage  this  phase.  During  phase 
1,  which  FAA  plans  to  complete  by  the  end  of  calendar  year  2002,  the 
agency  has  been  deploying  five  new  technologies  to  a  limited  number  of 
locations  and  measuring  their  benefits.  Figure  1  shows  how  these  five 
technologies — Surface  Movement  Advisor  (SMA),  User  Request 
Evaluation  Tool  (URET),  Traffic  Management  Advisor  (TMA), 
Collaborative  Decision  Making  (CDM),  and  passive  Final  Approach 
Spacing  Tool  (pFAST) — operate  to  help  manage  air  traffic.  According  to 
FAA,  SMA  and  CDM  have  been  deployed  at  all  phase  1  sites  on  or  ahead  of 
schedule.  Table  1  shows  FAA’s  actual  and  planned  deployment  dates  for 
URET,  TMA,  and  pFAST. 
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Figure  1 :  Use  of  Free  Flight’s  Tools  to  Help  Manage  Air  Traffic 


SMA  helps  airlines  more 
efficiently  manage  aircraft 
movement  on  the  ground. 


URET  helps  FAA's  en  route 
controllers  automatically  identify 
conflicts  and  respond  to  pilots' 
requests  for  route  changes  in  the 
en  route  airspace.3 


TMA  helps  FAA's  en  route  controllers  and  traffic 
managers  more  efficiently  schedule  properly 
separated  aircraft  as  they  transition  into  the 
terminal  airspace  around  capacity-constrained 
airports.*3 


URET 


TMA 


SMA 


CDM 


CDM  helps  FAA  and  the  airlines  electronically 
exchange  and  analyze  information  on  flight 
status  and  airspace  capacity. 


pFAST 


pFAST  helps  FAA's  terminal  controllers  more 
efficiently  schedule  the  final  sequencing  and 
assign  runways  to  aircraft  in  the  terminal 
airspace. 


aEn  route  airspace  usually  extends  above  18,000  feet  for  commercial  aircraft. 

bTerminal  airspace  begins  about  5  miles  and  ends  about  50  miles  from  the  airport  and  generally  extends  up  to  10,000  feet  above  the  ground. 

Source:  FAA. 
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Table  1 :  Phase  1  Deployment  Dates  for  TMA,  pFAST,  and  URET 


Location 

TMA 

pFAST 

URET 

Dallas-Fort  Worth 

June  1996a 

February  1999a 

Minneapolis 

June  2000 

June  2001 

Denver 

September  2000 

Los  Angeles 

November  2000 

February  2001 

Atlanta 

February  2001 

March  2001 

February  2002 

Miami 

May  2001 

Oakland 

September  2001 

Saint  Louis 

October  2001 

Indianapolis 

December  2001 b 

Memphis 

November  2001 b 

Kansas  City 

December  2001 

Cleveland 

January  2002 

Washington,  D.C. 

January  2002 

Chicago 

February  2002 

Prototypes  of  TMA  and  pFAST  (not  the  actual  models  to  be  deployed)  are  being  used. 


bA  prototype  of  URET  (not  the  actual  model  to  be  deployed)  has  been  used  at  these  locations  since 
November  1997. 

Source:  GAO’s  presentation  of  FAA’s  data. 

To  measure  whether  the  free  flight  tools  will  increase  system  capacity  and 
efficiency,  in  phase  1,  FAA  has  been  collecting  data  for  the  year  prior  to 
deployment  and  initially  planned  to  collect  this  information  for  the  year 
after  deployment  before  making  a  decision  about  moving  forward.  In 
December  1999,  at  the  urging  of  the  aviation  community,  FAA  accelerated 
its  funding  request  to  enable  it  to  complete  the  next  phase  of  the  free  flight 
program  by  2005 — 2  years  ahead  of  schedule.  During  this  second  phase, 
FAA  plans  to  deploy  some  of  the  tools  at  additional  locations  and  colocate 
some  of  them  at  selected  facilities.5  FAA  also  plans  to  conduct  research 
on  enhancements  to  these  tools  and  incorporate  them  when  they  are 
sufficiently  mature.  FAA  plans  to  make  an  investment  decision  in  March 
2002  about  whether  to  proceed  to  phase  2.  However,  by  that  date,  the  last 
site  for  URET  will  have  been  operational  for  only  1  month,  thus  not 
allowing  the  agency  to  collect  data  for  1  year  after  deployment  for  that  site 
before  deciding  to  move  forward.  (See  table  1.)  FAA  officials  told  us  that 


5In  May  2000,  FAA  named  the  director  for  Free  Flight  Phase  2,  during  which  FAA  plans  to 
deploy  the  tools  at  fewer  locations  than  the  aviation  industry  had  recommended  because  of 
technical  challenges  with  selected  tools,  infrastructure  limitations,  and  other  budgetary 
priorities. 
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because  the  preliminary  data  showed  that  the  benefits  were  occurring 
more  rapidly  than  anticipated,  they  believe  it  is  unnecessary  to  wait  for  the 
results  from  the  evaluation  plan  to  make  a  decision  about  moving  forward. 

To  help  airports  achieve  their  maximum  capacity  for  arrivals  through  free 
flight,  FAA’s  controllers  will  undergo  a  major  cultural  change  in  how  they 
will  manage  the  flow  of  air  traffic  over  a  fixed  point  (known  as  metering). 
Under  the  commonly  used  method,  controllers  use  “distance”  to  meter 
aircraft.  With  the  introduction  of  TMA,  controllers  will  have  to  adapt  to 
using  “time”  to  meter  aircraft. 


URET’s  Major 
Technical  Challenge  Is 
Making  It  Work  With 
Other  Systems 


The  major  technical  challenge  with  deploying  the  free  flight  tools  is 
making  URET  work  with  FAA’s  other  air  traffic  control  systems.  While 
FAA  does  not  think  this  challenge  is  insurmountable,  we  believe  it  is 
important  for  FAA  to  resolve  this  issue  to  fully  realize  URET's  benefit  of 
increasing  controller  productivity.  Initially,  controllers  had  expressed 
concern  about  how  often  they  could  rely  on  TMA  to  provide  the  data 
needed  to  effectively  manage  the  flow  of  traffic.  However,  according  to 
FAA  and  subsequent  conversations  with  controllers,  this  problem  was 
corrected  in  May  2001  when  the  agency  upgraded  TMA  software  and 
deployed  the  new  version  to  all  sites.  To  FAA’s  credit,  it  has  decided  not 
to  deploy  pFAST  to  additional  facilities  in  phase  2  because  of  technical 
difficulties  associated  with  customizing  the  tool  to  meet  the  specific  needs 
of  each  facility,  designing  other  automated  systems  that  are  needed  to 
make  it  work,  and  affordability  considerations. 


Ensuring  that  URET  is  compatible  with  other  major  air  traffic  control 
systems  is  a  crucial  technical  challenge  because  this  requires  FAA  to 
integrate  software  changes  among  multiple  systems.  Among  these 
systems  are  FAA’s  HOST,  Display  System  Replacement,6  and  local 
communications  networks.  Compounding  this  challenge,  FAA  has  been 
simultaneously  upgrading  these  systems’  software  to  increase  their 
capabilities.  How  well  URET  will  work  with  these  systems  is  unknown 
because  FAA  has  yet  to  test  this  tool  with  them.  FAA  has  developed  the 
software  needed  for  integration  and  has  begun  preliminary  testing. 


6HOST  is  the  en  route  centers’  system  for  processing  flight  and  radar  data  that  are 
displayed  on  the  controllers’  workstations.  The  Display  System  Replacement  is  the  en  route 
workstation  that  graphically  displays  the  flight  plan  and  radar  data  processed  by  the  HOST 
computer.  The  Display  System  Replacement  replaces  20-  to  30-year  old  workstations  and 
provides  a  platform  for  transitions  to  other  air  traffic  control  enhancements. 
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Although  problems  have  been  uncovered  during  testing,  FAA  has  indicated 
that  these  problems  should  not  preclude  URET’s  continued  deployment. 

By  the  end  of  August  2001,  FAA  expects  to  complete  testing  of  URET’s 
initial  software  in  cor\junction  with  the  agency’s  other  major  air  traffic 
control  systems.  FAA  acknowledges  that  further  testing  might  uncover 
the  need  for  additional  software  modifications,  which  could  increase  costs 
above  FAA’s  current  estimate  for  this  tool’s  software  development  and 
could  cause  the  agency  to  defer  capabilities  planned  for  phase  1. 

Ensuring  URET’s  compatibility  with  other  air  traffic  control  systems  is 
important  to  fully  realize  its  benefits  of  increasing  controllers’ 
productivity.  URET  is  used  in  facilities  that  control  air  traffic  at  high 
altitudes  and  will  help  associate  and  lead  controllers  work  together  to 
safely  separate  aircraft.  Traditionally,  an  associate  controller  has  used  the 
data  on  aircraft  positions  provided  by  the  HOST  computer  and  displayed 
on  the  Display  System  Replacement  workstation  to  assess  whether  a 
potential  conflict  between  aircraft  exists.  If  so,  an  associate  controller 
would  annotate  the  paper  flight  strips  containing  information  on  their 
flights  and  forward  these  paper  flight  strips  to  the  lead  controller  who 
would  use  the  Display  System  Replacement  workstation  to  enter  flight 
plan  amendments  into  the  HOST.  URET  users  we  spoke  with  said  that  this 
traditional  approach  is  a  labor-intensive  process,  requiring  over  30 
keystrokes.  With  URET,  an  associate  controller  can  rely  on  this  tool  to 
automatically  search  for  potential  conflicts  between  aircraft,  which  are 
then  displayed.  URET  also  helps  an  associate  controller  resolve  a 
potential  conflict  by  automatically  calculating  the  implications  of  any 
change  prior  to  amending  the  flight  plan  directly  into  the  HOST.  According 
to  the  users  we  spoke  with,  these  amendments  require  only  three 
keystrokes  with  URET. 
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While  FAA’s  Phased 
Approach  Is 
Appropriate,  Adapting 
to  the  Cultural 
Changes  and 
Improving 

Communications  Are 
Major  Operational 
Challenges 


FAA,  controllers,  maintenance  technicians,  the  aviation  community,  and 
other  stakeholders  agree  on  the  importance  of  using  a  phased  approach  to 
implementing  the  free  flight  program.  This  approach  allows  FAA  the 
opportunity  to  gradually  deploy  the  new  technologies  at  selected  facilities 
and  users  to  gain  operational  experience  before  total  commitment  to  the 
free  flight  tools.  It  basically  follows  the  “build  a  little,  test  a  little,  field  a 
little”  approach  that  we  have  endorsed  on  numerous  occasions.  To  FAA’s 
credit,  the  agency  has  appropriately  used  this  approach  to  determine  that 
it  wifl  not  deploy  pFAST  in  phase  2.  We  also  agree  with  major 
stakeholders  that  adapting  to  the  program’s  tools  poses  the  greatest 
operational  challenge  because  they  will  change  the  roles  and 
responsibilities  of  the  controllers  and  others  involved  in  air  traffic 
services.  However,  the  success  of  free  flight  will  rely  on  agencywide 
cultural  changes,  especially  with  controllers,  who  trust  their  own 
judgment  more  than  some  of  FAA’s  new  technologies,  particularly  because 
the  agency’s  prior  efforts  to  deploy  them  have  had  significant  problems.7 
Without  training  in  these  new  tools,  air  traffic  controllers  would  be 
hampered  in  fulfilling  their  new  roles  and  responsibilities.  Another  major 
challenge  is  effectively  communicating  TMA’s  capabilities  to  users. 
Because  FAA  has  been  deferring  and  changing  capabilities,  it  has  been 
difficult  for  controllers  to  know  what  to  expect  and  when  from  this  tool 
and  for  FAA  to  ensure  that  it  provides  all  the  capabilities  that  had  been 
agreed  when  FAA  approved  the  investment  for  phase  1. 


The  Roles, 
Responsibilities,  and 
Methods  for  Air  Traffic 
Control  Will  Change 


During  our  meetings  with  air  traffic  controllers  and  supervisors,  their 
biggest  concern  was  that  the  free  flight  tools  would  require  cultural 
changes  in  the  way  they  carry  out  their  responsibilities.  By  increasing 
their  dependence  on  automation  for  their  decisionmaking,  these  tools  are 
expected  to  help  increase  controllers’  productivity.  Moreover,  the  tools 
will  require  changes  in  commonly  recognized  and  accepted  methods  for 
managing  traffic. 


7Key  systems  under  FAA’s  air  traffic  control  modernization  program,  of  which  the  free 
flight  program  is  a  major  part,  have  often  had  difficulty  in  meeting  cost,  schedule,  and 
performance  requirements.  Because  of  the  size,  complexity,  cost,  and  problem-plagued 
past  of  FAA’s  modernization  program,  we  have  designated  it  a  high-risk  information 
technology  investment  since  1995.  FAA’s  modernization  program  is  one  of  four  high-risk 
systems  development  and  modernization  efforts.  See  High-Risk  Senes:  An  Overview 
(GAO/HR-95-1,  Feb.  1995);  High-Risk  Series:  Information  Management  and  Technology 
(GAO/HR-97-9,  Feb.  1997);  High-Risk  Senes:  An  Update  (GAO/HR-99-1,  Jan.  1999);  and 
High-Risk  Series:  An  Update  (GAO-Ol-263,  Jan.  2001). 
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Controllers  and  supervisors  emphasized  that  URET  will  increase  the 
responsibilities  of  the  associate  controllers  in  two  important  ways.  First, 
their  role  would  no  longer  be  focused  primarily  on  separating  traffic  by 
reading  information  on  aircraft  routes  and  altitudes  from  paper  flight 
strips,  calculating  potential  conflicts,  and  manually  reconfiguring  the 
strips  in  a  tray  to  convey  this  information  to  a  lead  controller.  With  the 
URET  software  that  automatically  identifies  potential  conflicts  up  to  20 
minutes  in  advance,  associate  controllers  can  be  more  productive  because 
they  will  no  longer  have  to  perform  these  manual  tasks.  Second,  they  can 
assume  a  more  strategic  outlook  by  becoming  more  focused  on  improving 
the  use  of  the  airspace.  URET  enables  them  to  be  more  responsive  to  a 
pilot’s  request  to  amend  a  flight  plan  (such  as  to  take  advantage  of 
favorable  winds)  because  automation  enables  them  to  more  quickly  check 
for  potential  conflicts  before  granting  a  request.  Although  the  controllers 
said  they  look  forward  to  assuming  this  greater  role  and  believe  that  URET 
will  improve  the  operational  efficiency  of  our  nation’s  airspace,  they  have 
some  reservations. 

Achieving  this  operational  efficiency  comes  with  its  own  set  of  cultural 
and  operational  challenges.  Culturally,  controllers  will  have  to  reduce 
their  dependency  on  paper  flight  strips  as  URET  presents  data 
electronically  on  a  computer  screen.  According  to  the  controllers  we 
interviewed,  this  change  will  be  very  challenging,  especially  at  facilities 
that  handle  large  volumes  of  traffic,  such  as  Chicago,  because  the  two 
facilities  that  have  received  URET  have  taken  several  years  to  become 
proficient  with  it  even  though  they  have  less  traffic.  Operationally, 
controllers  said  that  URET’s  design  must  include  some  backup  capability 
because  they  foresee  the  tool  becoming  a  critical  component  in  future 
operations.  Moreover,  as  controllers  become  increasingly  experienced 
and  reliant  on  URET,  they  will  be  reluctant  to  return  to  the  former  manual 
way  because  those  skills  will  have  become  less  current.  As  new 
controllers  join  the  workforce,  an  automated  backup  capability  will 
become  increasingly  essential  because  they  will  not  be  familiar  with 
controlling  traffic  manually  with  paper  flight  strips.  Currently,  FAA  is  not 
committed  to  providing  a  backup  to  URET  in  either  phase  because  the  tool 
is  only  a  support  tool,  not  a  mission-critical  tool  that  requires  backup. 
However,  the  agency  is  taking  preliminary  steps  to  provide  some 
additional  space  for  new  equipment  in  the  event  it  decides  to  provide  this 
backup.  Depending  on  how  the  agency  plans  to  address  this  issue,  the  cost 
increase  will  vary. 

For  TMA,  controllers  emphasized  during  our  discussions  that  using  time 
rather  than  distance  to  meter  properly  separated  aircraft  represents  a 
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major  cultural  shift.  While  controllers  can  visually  measure  distance,  they 
cannot  do  the  same  with  time.  As  one  controller  in  a  discussion  group 
commented,  TMA  “is  going  to  be  a  strain,  . . .  and  I  hate  to  use  the  word 
sell,  but  it  will  be  a  sell  for  the  workforce  to  get  this  on  the  floor  and  turn  it 
on  and  use  it.”  Currently,  controllers  at  most  en  route  facilities  use 
distance  to  meter  aircraft  as  they  begin  their  descent  into  an  airport’s 
terminal  airspace.  This  method,  which  relies  on  the  controllers’  judgment, 
results  in  the  less  efficient  use  of  this  airspace  because  controllers  often 
add  distance  between  planes  to  increase  the  margin  of  safety.  With  TMA, 
controllers  will  rely  on  the  computer’s  software  to  assign  a  certain  time  for 
aircraft  to  arrive  at  a  predetermined  point.  Through  continuous  automatic 
updating  of  its  calculations,  TMA  helps  balance  the  flow  of  arriving  flights 
into  the  congested  terminal  airspace  by  rapidly  responding  to  changing 
conditions.  The  controllers  at  the  first  three  of  the  en  route  centers  that 
have  transitioned  to  TMA  easily  accepted  it  because  they  had  been  using 
time  to  meter  air  traffic  for  20  years.  However,  as  other  en  route  centers 
transition  to  TMA,  the  controllers’  receptivity  will  be  difficult  because  they 
have  traditionally  used  distance  to  meter  air  traffic.  FAA  management 
realizes  that  the  controllers’  transition  to  metering  based  on  time  versus 
distance  will  be  challenging  and  has  allowed  at  least  1  full  year  for  them  to 
become  proficient  in  using  the  tool  and  begin  to  reap  its  full  benefits.  As  a 
result,  the  Free  Flight  Program  Office  has  established  a  1-year  period  for 
controllers  to  become  trained  and  comfortable  with  using  this  tool. 


Controllers  Believe  That 
Their  Efforts  to  Provide 
Adequate  Training  Have 
Been  Hampered 


FAA  is  relying  heavily  on  national  user  teams  to  help  develop  training  for 
TMA  and  URET.8  However,  because  of  a  lack  of  training  development 
expertise  and  other  factors,  their  efforts  to  provide  adequate  training  for 
TMA  have  been  hampered.  Controllers  said  that,  while  they  have 
knowledge  of  TMA,  they  are  not  specialists  in  developing  training  and 
therefore  need  more  assistance  from  the  program  office.  Also,  because 
only  a  few  key  controllers  have  experience  in  using  TMA,  the  teams  have 
had  to  rely  on  them  to  develop  a  standardized  training  program  while 
working  with  local  facilities  to  tailor  it  to  their  needs.  Moreover,  these 
controllers  are  being  asked  to  troubleshoot  technical  problems.  Finally, 
controllers  said  the  computer-based  training  they  have  received  to  date 
has  not  been  effective  because  it  does  not  realistically  simulate 
operational  conditions.  FAA  is  currently  revising  its  computer-based 
training  to  provide  more  realistic  simulations. 


8These  teams  are  comprised  of  controllers,  technicians,  and  supervisors. 


Page  12 


GAO-0 1-93 2  FAA’s  Free  Flight  Program 


Because  using  the  free  flight  tools  will  require  controllers  to  undergo  a 
complex  and  time-consuming  cultural  change,9  developing  a 
comprehensive  training  program  would  greatly  help  FAA’s  efforts  to 
implement  the  new  free  flight  technologies.  Communicating  to  users  how 
the  new  tools  will  benefit  the  organization  and  them  will  greatly  enhance 
the  agency’s  training  strategy.  While  FAA’s  training  plans  for  URET  are 
preliminary  because  it  is  undergoing  testing  and  is  not  scheduled  for 
deployment  until  the  latter  part  of  2001,  we  believe  that  providing 
adequate  training  in  advance  is  essential  for  controllers  to  become 
proficient  in  using  this  tool. 


Lack  of  Communication  on 
TMA’s  Capabilities  Leaves 
Users  Uncertain  About  the 
Tool’s  Benefits 


Our  discussions  with  controllers  and  FAA’s  TMA  contractor  indicated  that 
in  order  to  address  local  needs  and  to  fix  technical  problems  with  TMA, 
FAA  deferred  several  aspects  of  the  tool  that  had  been  established  for 
earlier  deployment  in  phase  1.  FAA  officials  maintain  that  these 
capabilities  will  be  deployed  before  the  end  of  phase  1.  However,  if  these 
capabilities  are  not  implemented  in  phase  1,  pushing  them  into  phase  2 
will  likely  increase  costs  and  defer  benefits.  For  example,  TMA’s  full 
capability  to  process  data  from  adjacent  en  route  centers  has  been 
changed  because  FAA  determined  that  providing  the  full  capability  was 
not  cost  effective.  While  controllers  said  that  even  without  this  full 
capability,  TMA  has  provided  some  benefits,  they  said  that  deferring  some 
aspects  of  the  tool’s  capabilities  has  made  it  less  useful  than  they 
expected.  Moreover,  controllers  maintain  that  FAA  has  not  clearly 
communicated  the  changes  with  the  tool’s  capabilities  to  them.  Without 
knowing  how  the  tool’s  capabilities  are  being  changed  and  when  the 
changes  will  be  incorporated,  it  is  difficult  for  users  to  know  what  to 
expect  and  when  and  for  FAA  to  evaluate  the  tool’s  cost,  schedule,  and 
ability  to  provide  expected  benefits. 


9For  a  discussion  on  organizational  culture  within  FAA,  see  Aviation  Acquisition:  A 
Comprehensive  Strategy  Is  Needed  for  Cultural  Change  at  FAA  (GAO/RCED-96-159,  Aug. 
22,  1996)  and  Major  Management  Challenges  and  Program  Risks:  Department  of 
Transportation  (GAO-Ol-253,  Jan.  2001). 
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Preliminary  Data 
Indicate  That  the  Free 
Flight  Tools  Will 
Probably  Increase 
Capacity  and 
Efficiency  Thereby 
Slowing  the  Growth  in 
Delays 

Early  Results  Show 
Capacity  and  Efficiency 
Increases 


FAA  has  begun  to  measure  capacity  and  efficiency  gains  from  using  the 
free  flight  tools  and  its  preliminary  data  show  that  the  tools  provide 
benefits.  FAA  expects  additional  sites  to  show  similar  or  greater  benefits, 
thus  providing  data  to  support  a  decision  to  move  to  phase  2  by  March 
2002.  Because  the  future  demand  for  air  traffic  services  is  expected  to 
outpace  the  tools’  capacity  increases,  the  collective  length  of  delays  during 
peak  periods  will  continue  to  increase  but  not  to  the  extent  that  they 
would  have  without  them. 


When  FAA,  in  collaboration  with  the  aviation  industry,  instituted  the 
phased  approach  to  implement  its  free  flight  program  in  1998,  the  agency 
established  a  qualitative  goal  for  increasing  capacity  and  efficiency.  In 
May  2001,  FAA  announced  quantifiable  goals  for  each  of  the  three  tools. 
For  URET,  FAA  established  an  efficiency  goal  to  increase  direct  routings 
by  15  percent  within  the  first  year  of  being  fully  implemented.  Achieving 
this  goal  translates  into  reduced  flight  times  and  fuel  costs  for  the  airlines. 
The  capacity  goals  for  TMA  and  pFAST  are  dependent  upon  whether  they 
are  used  together  (colocated)  and  whether  any  constraints  at  an  airport 
prevent  them  from  being  used  to  their  full  potential  to  expand  capacity.  If 
they  are  used  together  (such  as  at  Minneapolis),  FAA  expects  capacity  to 
increase  by  3  percent  in  the  first  year  of  operations  and  by  5  percent  in  the 
following  year.  However,  at  Atlanta,  which  is  constrained  by  a  lack  of 
runways,  the  goal  is  3  percent  when  these  tools  are  used  together.  If  only 
one  of  these  tools  is  deployed  (such  as  at  Miami),  FAA  expects  a  3-percent 
increase  in  capacity.  While  FAA  has  established  quantifiable  goals  for 
these  tools,  the  agency  has  only  recently  begun  to  develop  information  to 
determine  whether  attaining  its  goals  will  result  in  a  positive  return  on  the 
investment.  Making  this  determination  is  important  to  help  ensure  that  the 
capacity  and  efficiency  gains  provided  by  these  tools  are  worth  the 
investment. 

As  previously  shown  in  table  1,  the  actual  systems  that  will  be  deployed 
for  TMA  and  pFAST  have  only  recently  been  installed  at  several  locations 
or  are  scheduled  to  be  installed  this  winter.  To  date,  prototypes  of  these 
tools  have  been  colocated  at  one  location,  and  the  actual  equipment  has 
been  colocated  at  three  locations.  TMA  is  in  a  stand-alone  mode  at  two 
locations.  FAA  reported  that  TMA  achieved  its  first-year  goal  of  a  3- 
percent  increase  in  capacity  at  Minneapolis,  and  the  agency  is  collecting 
data  to  determine  whether  the  tool  is  meeting  its  goals  at  the  other 
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locations.  Most  of  FAA’s  data  regarding  the  benefits  provided  by  these 
tools  are  based  on  operations  of  their  prototypes  at  Dallas-Fort  Worth. 
These  data  show  that  TMA  and  pFAST  achieved  the  5-percent  colocation 
goal.  However,  the  data  might  not  be  indicative  of  the  performance  of  the 
actual  tools  that  will  be  deployed  to  other  locations  because  Dallas-Fort 
Worth  does  not  face  the  constraints  affecting  many  other  airports  (such  as 
a  lack  of  runways). 

Because  FAA  does  not  plan  to  begin  deploying  the  actual  model  of  URET 
until  November  2001,  the  agency's  data  on  its  benefits  have  been  based 
only  on  a  prototype.  At  the  two  facilities — Indianapolis  and  Memphis — 
where  the  prototype  has  been  deployed  since  1997,  FAA  reported  that 
URET  has  increased  the  number  of  direct  routings  by  over  17  percent  as  of 
April  2001.  According  to  FAA’s  data,  all  flights  through  these  two  facilities 
were  shortened  by  an  average  of  one-half  mile,  which  collectively  saved 
the  airlines  approximately  $1.5  million  per  month  in  operating  costs. 
However,  the  benefits  that  FAA  has  documented  for  using  URET  reflect 
savings  for  just  a  segment  of  a  flight — when  an  airplane  is  cruising  through 
high-altitude  airspace— not  the  entire  flight  from  departure  to  arrival. 
Maintaining  URET’s  benefits  for  an  entire  flight  is  partly  dependent  on 
using  it  in  conjunction  with  TMA  and  pFAST.  Although  a  researcher  at  the 
Massachusetts  Institute  of  Technology,  who  is  reviewing  aspects  of  FAA’s 
free  flight  program,  recognizes  URET’s  potential  benefits,  the  researcher 
expressed  concerns  that  its  benefits  could  be  lessened  in  the  airspace 
around  airports  whose  capacity  is  already  constrained.  Likewise,  in  a 
study  on  free  flight  supported  by  the  National  Academy  of  Sciences  and 
the  Department  of  Transportation,  the  authors  found  that  the  savings 
attributed  to  using  direct  routings  might  “be  lost  as  a  large  stack  of  rapidly 
arriving  aircraft  must  now  wait”  in  the  terminal  airspace  at  constrained 
airports.10  Although  URET  can  get  an  airplane  closer  to  its  final 
destination  faster,  airport  congestion  will  delay  its  landing.  While  TMA 
and  pFAST  are  designed  to  help  an  airport  handle  arrivals  more  efficiently 
and  effectively,  they  cannot  increase  the  capacity  of  an  airport’s  terminal 
airspace  beyond  the  physical  limitations  imposed  by  such  constraining 
factors  as  insufficient  runways  or  gates. 

In  contrast,  FAA’s  Free  Flight  Program  Office  believes  that  the  savings 
observed  with  the  prototype  of  URET  will  accrue  when  the  actual  tool  is 


10Christopher  D.  Wickens  and  Anne  S.  Mavor,  et  al.,  The  Future  of  Mr  Traffic  Control: 
Human  Operators  and  Automation ,  Commission  on  Behavioral  and  Social  Sciences  and 
Education,  National  Research  Council,  National  Academy  Press,  1998. 
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used  in  conjunction  with  TMA  and  pFAST.  FAA  plans  to  have  procedures 
in  place  by  the  time  these  three  tools  are  used  together  so  that  URET’s 
benefits  will  not  be  reduced.  However,  the  colocation  of  these  three  tools 
is  not  expected  to  occur  until  February  2002,  which  is  only  1  month  before 
the  agency  plans  to  make  an  investment  decision  for  phase  2.  Thus,  we 
believe  that  FAA  will  not  have  enough  time  to  know  whether  URET’s 
benefits  would  be  reduced. 


Free  Flight  Tools  Are 
Expected  to  Slow  the 
Growth  in  Air  Traffic 
Delays 


During  peak  periods,  the  demand  for  air  traffic  currently  exceeds  capacity 
at  some  airports,  causing  delays.  FAA  expects  this  demand  to  grow, 
meaning  that  more  aircraft  will  be  delayed  for  longer  periods.  Free  flight 
tools  have  the  potential  to  allow  the  air  traffic  system  to  handle  more 
aircraft  (increase  capacity)  but  not  to  keep  up  with  the  projected  growth 
in  demand.  Thus,  they  can  only  slow  the  growth  of  future  delays.  They 
cannot  fully  eliminate  future  delays  or  reduce  current  delays  unless 
demand  remains  constant  or  declines.  FAA’s  model  of  aircraft  arrivals  at  a 
hypothetical  congested  airport,  depicted  in  figure  2,  illustrates  the 
projected  impact  of  the  tools.  According  to  the  model,  if  demand 
increases  and  the  tools  are  not  deployed  (capacity  remains  constant);  the 
collective  delays  for  all  arriving  flights  (not  each  one)  will  increase  by 
about  an  hour  during  peak  periods.  But  if  demand  increases  exceed 
capacity  increases  from  deploying  the  tools,  these  delays  will  only 
increase  by  about  half  an  hour. 
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Figure  2:  Relationships  Among  Demand,  Capacity,  and  Delay 
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Source:  GAO’s  conceptual  presentation  of  FAA’s  model. 


FAA  Has  Noted  Other 
Benefits  From  the  Free 
Flight  Tools 


While  recognizing  that  the  free  flight  tools  will  provide  other  benefits,  FAA 
has  not  quantified  them.  According  to  FAA,  although  TMA  and  pFAST  are 
designed  to  maximize  an  airport’s  arrival  rates,  they  also  can  increase 
departure  rates  because  of  their  ability  to  optimize  the  use  of  the  airspace 
and  infrastructure  around  an  airport.  Regarding  URET,  FAA  maintains  that 
by  automating  some  of  the  functions  that  controllers  had  performed 
manually,  such  as  manipulating  paper  flight  strips,  the  tool  allows 
controllers  to  be  more  productive. 


Conclusions 


If  FAA’s  data  continue  to  show  positive  benefits,  the  agency  should  be  in  a 
position  by  March  2002  to  make  a  decision  to  deploy  TMA  to  additional 
sites.  However,  FAA  might  not  be  in  a  position  to  make  an  informed 
decision  on  URET  because  the  schedule  might  not  allow  time  to  collect 
sufficient  data  to  fully  analyze  the  expected  benefits  from  this  tool  during 
phase  1. 
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Currently,  operational  issues  present  the  greatest  challenge  because  using 
the  free  flight  tools  will  entail  a  major  cultural  shift  for  controllers  as  their 
roles  and  responsibilities  and  methods  for  managing  air  traffic  will  change. 
While  FAA  management  has  recognized  the  cultural  changes  involved, 
they  have  not  taken  a  leadership  role  in  responding  to  the  magnitude  of 
the  changes.  In  particular,  while  involving  controllers  in  developing  and 
delivering  training  on  these  new  tools,  FAA  has  not  provided  support  to 
ensure  that  the  training  can  be  effectively  developed  and  presented  at 
local  sites.  Because  the  agency  has  been  changing  the  capabilities  of  TMA 
from  what  had  been  originally  planned  but  not  systematically  documenting 
and  communicating  these  changes,  FAA  and  the  users  of  this  tool  lack  a 
common  framework  for  understanding  what  is  to  be  accomplished  and 
whether  the  agency  has  met  its  goals. 

While  the  free  flight  tools  have  demonstrated  their  potential  to  increase 
capacity  and  save  the  airlines  money,  only  recently  has  FAA  established 
quantifiable  goals  for  each  tool  and  begun  to  determine  whether  its  goals 
are  reasonable— that  they  will  result  in  a  positive  return  on  investment. 
Because  several  factors  influence  the  benefits  expected  from  the  tools,  it 
is  important  for  FAA  to  clearly  articulate  the  expectations  for  each  tool  by 
specific  location. 


Recommendations 


To  make  the  most  informed  decision  about  moving  to  phase  2  of  the  free 
flight  program,  we  recommend  that  the  Secretary  of  Transportation  direct 
the  FAA  Administrator  to  take  the  following  actions: 

Collect  and  analyze  sufficient  data  in  phase  1  to  ensure  that  URET  can 
effectively  work  with  other  air  traffic  control  systems. 

Improve  the  development  and  the  provision  of  local  training  to  enable  field 
personnel  to  become  proficient  with  the  free  flight  tools. 

Determine  that  the  goals  established  in  phase  1  result  in  a  positive  return 
on  investment  and  collect  data  to  verify  that  the  goals  are  being  met  at 
each  location. 

Establish  a  detailed  set  of  capabilities  for  each  tool  at  each  location  for 
phase  2  and  establish  a  process  to  systematically  document  and 
communicate  changes  to  them  in  terms  of  cost,  schedule,  and  expected 
benefits. 
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Agency  Comments 


We  provided  a  draft  of  this  report  to  the  Department  of  Transportation  and 
the  National  Aeronautics  and  Space  Administration  for  their  review  and 
comment. 

We  met  with  officials  from  the  Office  of  the  Secretary  and  FAA,  including 
the  Director  and  Deputy  Director  Free  Flight  Program  Office,  to  obtain 
their  comments  on  the  draft  report.  These  officials  generally  concurred 
with  the  recommendations  in  the  draft  report.  They  stated  that,  to  date, 
FAA  has  completed  deployment  of  the  Surface  Movement  Advisor  and  the 
Collaborative  Decision  Making  tools  on,  or  ahead  of,  schedule  at  all 
phase  1  locations  and  plans  to  complete  the  deployment  of  the  remaining 
free  flight  tools  on  schedule.  FAA  officials  also  stated  that  the  agency  is 
confident  that  it  will  be  in  position  to  make  an  informed  decision,  as 
scheduled  in  March  2002,  about  moving  to  the  program’s  next  phase, 
which  includes  the  geographic  expansion  of  TMA  and  URET. 

Furthermore,  FAA  stated  that  the  free  flight  tools  have  already 
demonstrated  positive  benefits  in  an  operational  environment  and  that  it 
expects  these  benefits  will  continue  to  be  consistent  with  the  program’s 
goals  as  the  tools  are  installed  at  additional  sites.  In  addition,  FAA 
officials  provided  technical  clarifications,  which  we  have  incorporated  in 
this  report,  as  appropriate. 

We  acknowledge  that  FAA  has  deployed  the  Surface  Movement  Advisor 
and  the  Collaborative  Decision  Making  tools  on  schedule  at  various 
locations.  Furthermore,  the  report  acknowledges  that  the  free  flight  tools 
have  demonstrated  benefits  and  that  the  agency  should  have  the  data  on 
TMA  to  make  a  decision  about  moving  forward  to  phase  2  by  March  2002. 
However,  as  we  note  in  the  report,  FAA  faces  a  significant  technical 
challenge  in  ensuring  that  URET  works  with  other  air  traffic  control 
systems.  Moreover,  the  data  on  URET's  benefits  reflect  those  of  the 
prototype  system.  FAA  is  scheduled  to  deploy  the  first  actual  system  in 
November  2001  and  the  last  in  February  2002— just  1  month  before  it  plans 
to  make  an  investment  decision.  With  this  schedule,  the  actual  system 
might  not  be  operational  long  enough  to  gather  sufficient  data  to  measure 
its  benefits.  Furthermore,  FAA  has  yet  to  overcome  the  operational 
challenge  that  is  posed  when  controllers  use  TMA  and  must  shift  from  the 
traditional  distance-based  method  of  metering  air  traffic  to  one  based  on 
time.  If  FAA  can  not  satisfactorily  resolve  these  issues,  the  free  flight 
program  might  not  continue  to  show  positive  benefits  and  could 
experience  cost  overruns,  delays,  and  performance  shortfalls. 
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The  National  Aeronautics  and  Space  Administration  expressed  two  major 
concerns.  First,  it  felt  that  the  benefits  provided  from  the  TMA  tool 
justified  its  further  deployment.  Our  initial  conclusion  in  the  draft  report, 
that  FAA  lacked  sufficient  data  to  support  deploying  this  tool  to  additional 
sites,  was  based  on  FAA’s  initial  evaluation  plan,  which  required  at  least  1 
year  of  operational  data  after  each  tool  had  been  deployed.  FAA  officials 
now  believe  that  waiting  for  full  results  from  the  evaluation  plan  before 
making  a  decision  to  move  forward  is  no  longer  necessary  because  TMA's 
performance  results  are  occurring  more  rapidly  than  anticipated.  This 
report  now  acknowledges  that  the  agency  should  have  the  data  it  needs  to 
make  a  decision  to  move  forward  with  this  tool.  Second,  NASA  felt  that 
the  report  was  unclear  regarding  the  nature  of  our  concerns  about  the 
reliability  of  TMA's  data.  The  discussion  in  the  draft  report  indicated  that 
FAA  lacked  sufficient  data  to  show  that  it  had  addressed  our  concerns 
with  TMA.  FAA  officials  provided  this  support,  and  this  report  has  been 
revised  accordingly.  In  addition,  National  Aeronautics  and  Space 
Administration  officials  provided  technical  clarifications,  which  we  have 
incorporated  into  this  report,  as  appropriate.  (See  appendix  II  for  the 
National  Aeronautics  and  Space  Administration’s  comments.) 

As  agreed  with  your  offices,  unless  you  publicly  release  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  30  days  after  the 
date  of  this  letter.  At  that  time,  we  will  send  copies  of  this  report  to 
interested  Members  of  Congress;  the  Secretary  of  Transportation;  the 
Administrator,  Federal  Aviation  Administration;  and  the  Administrator, 
National  Aeronautics  and  Space  Administration.  We  will  also  make  copies 
available  to  others  upon  request. 

If  you  have  questions  about  this  report,  please  contact  me  at  (202)  512- 
3650.  Key  contributors  are  listed  in  appendix  III. 

Gerald  L.  Dillingham,  Ph.D. 

Director,  Physical  Infrastructure 
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Appendix  I:  Objectives,  Scope,  and 
Methodology 


Because  of  the  importance  of  the  free  flight  program  to  the  future 
operation  of  our  nation’s  aviation  system  and  the  upcoming  decision  about 
whether  to  proceed  to  the  next  phase,  the  Chairmen  of  the  Senate 
Committee  on  Commerce,  Science,  and  Transportation  and  the 
Subcommittee  on  Aviation  asked  us  to  provide  information  to  help  them 
determine  whether  the  Federal  Aviation  Administration  (FAA)  will  be  in  a 
position  to  decide  on  moving  to  the  next  phase.  This  report  discusses  (1) 
the  significant  technical  and  operational  issues  that  could  impair  the 
ability  of  the  free  flight  tools  to  achieve  their  full  potential  and  (2)  the 
extent  to  which  these  tools  will  increase  efficiency  and  capacity  while 
helping  to  minimize  delays  in  our  nation’s  airspace  system. 

Our  review  focused  on  three  free  flight  phase  1  tools — the  User  Request 
Evaluation  Tool,  the  Traffic  Management  Advisor,  and  the  passive  Final 
Approach  Spacing  Tool — because  they  account  for  approximately  80 
percent  of  FAA’s  $630  million  estimated  investment  for  phase  1  and 
approximately  80  percent  of  FAA’s  $717  million  estimated  investment  for 
phase  2.1  We  did  not  review  the  Surface  Movement  Advisor  or  the 
Collaborative  Decision  Making  tools  because  generally  they  had  been 
implemented  at  all  phase  1  locations  when  we  started  this  review  and  FAA 
does  not  intend  to  deploy  their  identical  functionality  in  phase  2. 

To  obtain  users’  insights  into  the  technical  and  operational  issues  and  the 
expected  benefits  from  these  tools,  we  held  four  formal  discussion  group 
meetings  with  nationwide  user  teams  made  up  of  controllers,  technicians, 
and  supervisors  from  all  the  facilities  currently  using  or  scheduled  to 
receive  the  Traffic  Management  Advisor  during  phase  1.  We  also  visited 
and/or  held  conference  calls  with  controllers,  technicians,  and  supervisors 
that  used  one  or  more  of  these  tools  in  Dallas,  Texas;  southern  California; 
Minneapolis,  Minnesota;  Memphis,  Tennessee;  Indianapolis,  Indiana;  and 
Kansas  City,  Kansas. 

Additionally,  to  determine  whether  there  were  any  significant  technical 
and  operational  issues,  we  considered  selected  best  practices  for 
developing,  managing,  and  evaluating  projects  from  the  Software 
Engineering  Institute’s  Capability  Maturity  Models  for  software 


lrThe  $630  million  estimated  investment  represents  development  costs  for  fiscal  years  1998 
through  2002,  and  the  $717  million  estimated  investment  represents  the  total  development 
costs  for  fiscal  years  2001  through  2010. 
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Methodology 


development  and  acquisition.2  Based  on  these  criteria,  we  interviewed 
FAA  officials  in  the  Free  Flight  Program  Office,  the  Office  of  Air  Traffic 
Planning  and  Procedures,  and  the  Office  of  Independent  Operational  Test 
and  Evaluation.  To  review  test  reports  and  other  documentation 
highlighting  technical  and  operational  issues  confronting  these  tools,  we 
visited  FAA’s  William  J.  Hughes  Technical  Center  in  Atlantic  City,  New 
Jersey,  and  FAA’s  prime  contractors  that  are  developing  the  three  free 
flight  tools.  We  also  visited  the  National  Aeronautics  and  Space 
Administration’s  Ames  Research  Center  at  Moffett  Field,  California,  to 
understand  how  its  early  efforts  to  develop  free  flight  tools  are  influencing 
FAA’s  current  enhancement  efforts. 

To  determine  the  extent  to  which  the  free  flight  tools  will  increase 
capacity  and  efficiency  while  helping  to  minimize  delays,  we  analyzed  the 
relevant  legislative  and  Office  of  Management  and  Budget’s  requirements 
that  recognize  the  need  for  agencies  to  develop  performance  goals  for 
their  mqjor  programs  and  activities.  We  also  interviewed  FAA  officials  in 
the  Free  Flight  Program  Office  and  the  Office  of  System  Architecture  and 
Investment  for  information  on  the  performance  goals  of  the  free  flight 
tools  during  phase  1.  In  addition,  we  held  discussions  with  officials  from 
RTCA,  which  provides  a  forum  for  government  and  industry  officials  to 
develop  consensus-based  recommendations.  We  also  reviewed 
documentation  explaining  how  the  tools  are  expected  to  and  actually  have 
helped  increase  system  capacity  and  efficiency,  thereby  helping  to 
minimize  delays. 

We  conducted  our  review  from  October  2000  through  July  2001,  in 
accordance  with  generally  accepted  government  auditing  standards. 


Capability  Maturity  Model  is  the  service  mark  of  Carnegie  Mellon  University  and  is 
registered  in  the  U.S.  Patent  and  Trademark  Office. 
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Appendix  II:  Comments  From  the  National 
Aeronautics  and  Space  Administration 


National  Aeronautics  and 
Space  Administration 
Office  of  the  Administrator 
Washington,  DC  20546-0001 


Dr.  Gerald  L.  Dillingham  ^ yg  0  2001 

Director,  Civil  Aviation  Issues 
General  Accounting  Office 
Washington,  DC  20548 


Subject :  GAO  Draft  Report:  National  Airspace  System:  Free  Flight  Tools  Hold 

Promise,  but  FAA  Will  Lack  Sufficient  Data  to  Decide  on  Moving  Forward 
(GAO-0 1-932) 


Thank  you  for  giving  us  the  opportunity  to  review  this  draft  report  and  provide  our 
comments  prior  to  final  publication.  We  have  had  individuals  at  Ames  Research  Center 
and  NASA  Headquarters  provide  input  into  our  comments.  This  response  covers  the 
major  points  for  your  records  as  the  formal  Agency  response. 

•  NASA  is  concerned  with  the  investment  conclusions  reached  by  the  report  based  on 
the  reported  technical  performance  of  Traffic  Management  Advisor  (TMA)  and 
passive  Final  Approach  Spacing  Tools  (pFAST).  (Reference  pages  14-16, 
“Preliminary  Data  Indicate  That  Free  Flight  Tools  Will  Probably  Increase  Capacity 
and  Efficiency,  Thereby  Slowing  the  Growth  in  Delays/’)  In  particular,  “Early 
Results  Show  Capacity  and  Efficiency  Increases,  but  Data  Arc  Inconclusive,” 
demonstrated  that  the  FAA  minimum  success  criteria  for  Minneapolis  and  Dallas-Fort 
Worth  were  all  exceeded  by  actual  performance  of  the  TMA  and  pFAST 
technologies.  Further,  in  “Free  Flight  Tools  Are  Expected  to  Slow  the  Growth  in  Air 
Traffic  Delays,”  the  report  indicates  that  the  decision-support  tools  will  slow  the 
growth  of  future  delays.  The  accompanying  figure,  “Relationship  Among  Demand, 
Capacity,  and  Delay”  shows  the  collective  effect  on  delays  with  the  decision-support 
tools  deployed,  as  well  as  when  the  tools  are  not  deployed.  More  specifically,  the 
figure  shows  that  if  demand  increases,  deploying  the  decision-support  tools  would  cut 
additional  delays  in  half.  It  appears  by  the  report’s  analysis  that  the  data  show  that 
the  tools  have  benefit  that  would  justify  a  conclusion  supporting  further  deployment 
with  appropriate  conditions  as  cited  elsewhere  in  the  report. 

•  A  more  specific  comment  addresses  the  concerns  regarding  the  accuracy  and 
reliability  of  TMA  aircraft  position  data.  On  page  8,  the  report  states  that  the  FAA 
claims  to  have  resolved  these  problems.  Consequently,  it  is  not  clear  if  there  is  still  a 
problem  or  not,  and  if  there  is  a  problem,  the  nature  of  the  problem.  Our 
understanding,  based  on  discussions  with  officials  in  the  Free  Flight  Office,  is  that  the 
“position  errors”  from  TMA  were  in  fact  due  to  a  fault  in  interface  logic  to  the  Host 
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computer.  This  fault  has  been  corrected  and  the  manifestation  of  that  problem  in 
TMA  no  longer  exists. 

•  It  is  important  to  understand  that  TMA  does  not  compute  instantaneous  aircraft 
position  data.  These  data  come  from  the  Host  and  are  used  by  TMA  to  predict  future 
aircraft  positions  in  the  absence  of  controller  intervention.  These  predictions  of 
position  are  then  used  by  TMA  to  recommend  scheduled  meter  fix  crossing  times  to 
assure  a  smooth  flow  of  traffic  into  the  (Terminal  Radar  Approach  Control) 
TRACON.  The  generated  schedules  can  be  judged  in  terms  of  the  ability  of  the 
controller  to  meet  the  scheduled  times,  not  their  accuracy.  Based  on  several  years  of 
testing,  NASA  has  found  the  TMA  trajectory  predictions  to  be  very  accurate  and  the 
schedules  to  be  very  useable  by  controllers.  This  is  consistent  with  the  controller 
observation  noted  in  your  report  that  “When  it’s  running  it’s  right  on  the  money.”  The 
ability  to  accurately  predict  future  aircraft  positions  is  fundamental  to  improve  the 
performance  of  the  air  transportation  system. 

If  you  require  any  further  assistance,  please  call  Herbert  Schtickcnmaier  on  358-4638. 


Daniel  R.  Mulville 
Associate  Deputy  Administrator 


R/Mr.  Venneri 
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